INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE 



OC 

C 

o 

(N 
U 



On the Almost Sure Central Limit Theorem for 
Vector Martingales : Convergence of Moments and 

Statistical Applications 



OC 



Bernard Bercu — Peggy Cenac — Guy FayoUe 



> 
OC 

(N 

in 
cn 

(N 

OC 
C 



TO 



N° 6780 

December 2008 
_ Theme NUM _ 




INSTITUT NATIONAL 
DE RECHERCHE 
EN INFORMATIQUE 
ET EN AUTOMATIQUE 




INK! A 



centre de recherche 

PARIS - ROCQUENCOURT 



On the Almost Sure Central Limit Theorem for 
Vector Martingales : Convergence of Moments and 
Statistical Applications 

Bernard Bercu □ , Peggy Cenac , Guy Fayollel 

Theme NUM — Systemes numeriques 
Equipe-Projet Imara 

Rapport de recherche n° 6780 — December 2008 — (23 pages 



Abstract: We investigate the almost sure asymptotic properties of vector 
martingale transforms. Assuming some appropriate regularity conditions 
both on the increasing process and on the moments of the martingale, we 
prove that normalized moments of any even order converge in the almost 
sure cental limit theorem for martingales. A conjecture about almost sure 
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are supported by examples borrowed from statistical applications, including 
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Theoreme de la limite centrale pour les martingales 
vectorielles : convergence des moments et 
applications statistiques 

Resume : On etudie dans ce rapport des proprietes de convergence presque 
sure de transformees de martingales vectorielles. Sous certaines conditions 
d'existence de moments et de regularite du processus croissant, on montre en 
particulier la convergence des moments normalises de tout ordre pair dans 
le theoreme central limite presque sur pour les martingales vectorielles. On 
formule une conjecture de borne presque siire, sous des hypotheses moins res- 
trictives et couvrant des families plus vastes de processus. Enfin, on applique 
ces resultats a des exemples issus d'applications statistiques, notamment les 
modeles autorcgressifs lineaires et certains processus de branchement avec 
immigration, ce qui permet d'etablir de nouvelles proprietes asymptotiques 
sur les erreurs d'estimation et de prediction. 

Mots-cles : Theoreme central limite presque sur, martingale vectorielle, 
moment, regression stochastique. 
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1 Introduction 



Let (Xn) be a sequence of real independent identically distributed random 
variables with E[X„] = and = cr^ and denote 

n 
k=l 

It follows from the ordinary central limit theorem (CLT) that 

■s/n 

which implies, for any bounded continuous real function h 



lim E 

n—^oo 



h 



Sn 

7i 



h{x)dG{x) 



where G stands for the Gaussian measure AA(0, cr^). Moreover, by the cele- 
brated almost sure central limit theorem (ASCLT), the empirical measure 

1 " 1 

Gn = \ V, 7*^^ 

logn k 

satisfies 

Gn G a.s. 
In other words, for any bounded continuous real function h 

lim -^Y^h(^] = [ h(x)dG(x) 



a.s. 



The ASCLT was simultaneously proved by Brosamler [3] and Schatte [15] 
and, in its present form, by Lacey and Phillip [10]. In contrast with the 
wide literature on the ASCLT for independent random variables, very few 
references are available on the ASCLT for martingales apart the recent work 
of Bercu and Fort [1,2] and the important contribution of Chaabane and 
Maaouia [4,5] and Lifshits [13,14]. The ASCLT for martingales is as follows. 
Let (e„) be a martingale difference sequence adapted to a filtration F = (Tn) 
with E[4+i|^n] = cr^ a.s. Let be a sequence of random variables 

adapted to F and denote by (M„) the real martingale transform 

n 
k=l 
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We also need to introduce the explosion coefficient fn associated with {^n) 

$2 " 
fn = — where Sn = Y^ ^i- 

k=o 

As soon as (/„) goes to zero a.s. and under reasonable assumption on the 
conditional momments of (sn), the ASCLT for martingales asserts that the 
empirical measure 

n 

Gn = T-—Y.ff'^^^G a.s. (1.1) 



In other words, for any bounded continuous real function h 



lim 

k=l 



.—y2fM^^)=[hix)dGix) a.s. (1.2) 



It is quite natural to overcome the case of unbounded functions h. To be more 
precisely, one might wonder if convergence (|1.2p remains true for unbounded 
functions h. It has been recently shown in [1,2] that, whenever (e^i) has a 
finite conditional moment of order > 2p , then the convergence p.2|) still 
holds for any continuous real function h such that \h{x)\ < x'^P. 

Theorem 1.1 (Convergence of moments in the scalar case). Assume that (e„) 
is a martingale difference sequence such that K[e'^_^_-^\J^n] = o"^ a.s. and 
satisfying for some integer p > 1 and for some real number a > 2p, 

supE[|e„_|_i|°'|jr„] < oo a.s. 

n>0 

If the explosion coefficient (fn) tends to zero a.s., then 



lim 



1 ^ , ( Mf.\p a'P{2p)\ , , 



n^oologs^^ \Sk-l^ 2Pp 



The limit (11.31) is exactly the moment of order 2p of the Gaussian distri- 
bution AA(0, a^). The purpose of the present paper is to extend the results 
of [1] to vector martingale transforms, which is strongly needed in various 
applications arising in statistics and signal processing. 

Let (M„) be a square integrable vector martingale with values in M*^, adapted 
to a filtration F. Its increasing process consists of the sequence ((M)„) of 
symmetric, positive semi-definite square matrices of order d given by 

n 

{M)n = ^E[(Mfc - Mk-i){Mk - Mk-if\J'k-i]. 

k=l 

INRIA 



Convergence of Moments in the ASCLT for Vector Martingales 



5 



A first version of the ASCLT for discrete vector martingales was proposed in 
[5,6], under fairly restrictive assumptions on the increasing process ((M)„). 
Hereafter, our goal is to establish the convergence of moments of even order in 
the ASCLT for (M„) under suitable assumptions on the behaviour of ((M)„). 
We shall work in the general setting of vector martingales transforms 
which can be written as 

n 
k=l 

where Mq can be taken arbitrary and ($„) denotes a sequence of random 
vectors of dimension d, adapted to F. We also introduce 

n 

Sn = ^'^k'^i + S (1.4) 

k=0 

where S is a fixed deterministic matrix, symmetric and positive definite. One 
can obviously see that if E[e^_|_]^|^„] = cr^ a.s., then the increasing process of 
(Mn) takes the form (M)„ = cj^5„_i. The explosion coefficient associated 
with (^n) is now given by 

fn = '^iS-^'^n = "/"-^ (1.5) 

where dn = det(S'„). 

After this short survey, the paper will be organized as follows. The main the- 
oretical result for vector martingale transforms is given in Section [2], at the 
end of which a quite plausible interesting conjecture is formulated, involving 
minimal assumptions. The final Section [3] proposes some statistical applica- 
tions to estimation and prediction errors in linear autoregressive models and 
in branching processes with immigration. 



2 On the convergence of moments 

As mentioned above, our main result is given in theorem 12.11 and extends 
theorem ll.ll to the vector case. By the way, in mathematics, the difficulty of 
the problem is almost always a strictly increasing function of the dimension 
of some underlying space: it is also the case here ! 

Theorem 2.1. Let (e^) be a martingale difference sequence satisfying the 
homogeneity condition E[e^_|_-^|^„] = o"^ a.s. and such that, for some integer 
p > 1 and some real number a > 2p, 

(H ) supE[|e„+i|"|j^„] < oo a.s. 

^ "P' n>0 
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In addition, assume that the explosion coefficient fn tends to zero a.s. and 
that there exists a positive random sequence (a„) increasing to infinity and 
an invertible symmetric matrix L, such that 

lim — Sn = L a.s. (2.1) 

n^oo an 

Then, the following limits hold almost surely 

n p-l 

ni^ W- E {MiS^-iM,y = i{p) = da'P ll{d + 2j) , (2.2) 

^ " k=l j=l 
1 " 

il^ E(^^^."-i^O" - {MiS^'M.y = Xip) = \l{p). (2.3) 

Remark 1. The limit £(p) corresponds exactly to the moment of order 2p of 
the norm of a gaussian vector AA(0, cr^/d), so that theorem 12.11 shows indeed 
the convergence of moments of order 2p in the ASCLT for vector martingales. 
Here the deterministic normalization given in [5] has been replaced by the 
natural random normalization given by the increasing process. 



Remark 2. The convergence hypothesis (|2.1|) clearly implies /n — > a.s., if 
and only if a„ ~ a„_i a.s. As a matter of fact, we deduce from (12. ip that 

lim — ^ = detL > a.s. 

Proof. For the sake of shortness, we shall define the following variables. 

Vn = MiS-\Mn. (2.4) 

Vn = a-\MiL-^Mn. 



First of all, by using the symmetry of L, the convergence (|2.ip ensures that 

fn = 'fn + o((^„) a.s. (2.5) 

Vn = Vn + o[Vn) a.S. (2.6) 

For we have 

/n = <^n + a^^^iL-^'\anL^'''S-^L^/^ - l)L-^'^^n. 

The matrix i?„ = a„-L^/^5~^L^/^ — / is symmetricand denoting by pn its 
spectral radius, we can write 



a-^^iL-^'^-RnL-^'^^r 



< Pn ^n, 
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and pn converges to almost surely, which leads to (|2.5I) . The equation (|2.6I) 
is proved in the same way from the decomposition 

Vn = vn + Mi{S-\ - a-\L-^)Mn. 

Hence, by (|2.6|) . Vn = Vn + o{y^ a.s. In order to find the limit (12. 2|) . il 
suffices, by Toeplitz lemma, to study the convergence 

ii- E = .1- E (2-7) 

fe=l fc=l 

Theorem 12.11 will be proved by induction with respect to p > 1, as in [1] 
in the scalar case. The first step consists in writing a recursive relation for 



ftj-i. 
Let 



Pn = tr(L / SnL ' ), Jn = ^ , 

Sn = run = f5-\MiL-^Mn. 

Pn 

According to the definition of (M^), the following decomposition holds 
so that 

m„+i = (1 - 7„)m„ + 2(5„e„+i + 7„e^+i. (2.9) 
The theorem relies essentially on the following lemma. 
Lemma 2.2. Under the assumptions of theorem \2.1\ we have 

lim V 7fcm^ = a.s. (2.10) 

n^oo log ^ dP+^ 

k=l 

In addition, if Qn = M^nS~}_i^n, we also have 

lim ^ X^(l - fk)9lml-' = ^ a.s. (2.11) 

n^oo log dn ^ vdP ^ 

k=l 

Proof. Raising equality (|2.9p to the power p implies 

<+i =EE2'"'^pC'f li6t'{{l-ln)mnY-'e'nti. (2.12) 

fc=0 £=0 

After some straightforward simplifications, we obtain the relation 

m^^i + An{p) =m{ + Bn+l{p) + Wn+l{p), (2.13) 
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where 

n 2p- 1 n 

k=l £=1 k=l 

n 

k=l 

For 1 < £ < p — 1, we have 
[e/2\ 

while, foi p < i < 2p — 1, 



j=i-{p-i) 



p-e+j 



In order to take out useful information about Anip), it is necessary to study 
the asymptotic behaviour }Vn+i{p), Sn+i{p) and rrin. 



The case p = 1. Remarking that log/3n ~ Sfc=i 7fc) Chow's lemma [7, page 
22] implies 



lim -^—Wn+i{l) = (T^ a.s. 

n^oo log fj^ 



Applying the strong law of large numbers for martingales and the inequality 
^ Infnm we get Bn+i{^) = 0(^,1(1)) a.s. In addition, from relation (2.30) 
in [17], it follows that m„+i = o(log/3„) a.s. Consequently, by (12.131) . 

lim -—^—An{l) = cf'^ a.s. 

n-+oo log fj^ 

But the basic convergence assumption (12. ip implies immediately 

so that log (in ~ dlogPn a.s. Hence 

lim ^ An{l) = ^ a.s., 

n-^oo log dn d 



which establishes (|2.10|) . 
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As for the proof of (|2.11l) . one can proceed in the same way, starting from 
the decomposition 

Vn+l = M'^S-^Mn + 2en+lKSn'^Mn + el^Jn 
= hn -\- 2en+l(jn + ^n+lfn- 

which, for all n > 1, leads to 

Vn+l +An = Vi + Bn+1 + Wn+l, (2.14) 

where 

n n n 

An = ^ 0^(1), Bn+1 = 2 ^ Ek+igk, Wn+1 = ^ £k+lfk- 
k=l k=l k=l 

By Riccati's formula, 

gl = {l- /„)a„(l), (2.15) 

and, hence, coupling (I2.15P with the strong law of large numbers for martin- 
gales, 

Bn+1 = o{An) a.s. 

On the other hand, m„ = o(log/3„), which, with (|2.6p . gives the almost sure 
estimate Vn = o(logdn). 

Now, since Y12=i fk ~ ^ogdn, Chow's lemma implies 
lim (logdn)'^Wn+i = fT^ a.s., 

n—>oo ' 

and to conclude the proof of Lemma [2?2l for p = 1, it suffices to divide (|2.14|1 
by logdn, letting n — > oo. 

The case p > 2. First, using again Chow's lemma, we can write 

Wn+i{p) = o(logdn) a.s. 
Also, we shall show that 

Bn+lip) = ^£{p)logdn + o{logdn) +o{An{p)) a.S. (2.16) 

Setting, for 1 < £ < 2p- 1, 

el+i = en+i{l) + E[4+i|.F„] = en+i{£) + a„(£), (2.17) 
we write Ylk=i ^fc(^)4+i = Cn+i(^) + ^?n(^), with 

n n 

Cn+i{i) = Y.^kii)ek+i{i), and Vnii) = J2^k{i)c7k{i)- 

k=l k=l 
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First, for any i such that 1 < ^ < p — 1, using again the strong law of large 
numbers and equation (2.30) of [17], we have 



Cn+l{t) = o{\ogdn 



a.s. 



n 

12 p-fJ2 



Suppose 3 < £ < p — 1. From Holder's inequality and the assumptions on 
the moments of it follows that, for all 1 < j < 2p — 1, \crnij)\ is bounded, 
which implies 

k=l 

For even i, the induction assumption leads to 'Dn{£) = o(logd„) a.s. When 
£ is odd, Cauchy Schwarz inequality and the induction assumption yield 

n n 
k=l k=l 

Suppose now p < £ < 2p — 1. It is easy to obtain 

n 



12 v-m^ 

k=l 

Now, from the induction assumption, we get, for any integer £ ^2, 

Vn{£) = o{[ogdn) a.s. 

It remains to study Cn+i{£)- By Chow's lemma, we have the almost sure 
equality 

n n 

Cn+i{£) = o{un{£)), With Mi) = Y.\bk{i)\''^' = 0['^jlm'^;^-'^^/'). 

k=l k=l 

For £ > p, we apply Holder's inequality with exponents £/p and £/{£ — p). 
Then, Vn{f) = o{\ogdn) a.s. In the particular case p = £, we get by the 
strong law of large numbers 

n n 

\Cn+li£)\^ = 0{Tn{p)l0gTn{p)) with Tn{p) = ^bk{pf = O (^"^jlml) , 

k=l k=l 

which leads to = o(logdn) a.s., since, from equation (2.30) of [17], 

= o((logd„)^), for < (5 < 1. As for the last term T>n{2), one needs to 
study, for p > 3, the quantity 



n 1 



logdn log-,. 



2p{p - l)a^ ^ 2 p-2 . Pc^ p-i 



k=l ° k=l 
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It is easy to establish 

v2 



al = {^iS-^.iMnY = <f5l + o(7nm„). (2.18) 
Then, the induction assumption and Toephtz lemma imply 



1 ^ \ \(p 1) 
lim ;- / <5frm?~ = lim — r- ;- / 0^(1)^-?" =7 ttt- a.s. 

n^oologdn^ n^oo (i2 log d„ ^ (p - l)(iP 

Thus, we obtain 

[this formula being also valid for p = 2] which proves (|2.16|) . 

On the other hand, still applying equation (2.30) of [17], we derive the esti- 
mate rrin = o(logd„). Thus, 

1 p 

n^oo log dn dP+^ 



Since 



we get finally 



^™ i — Tl^^kml = hm — — —Anij)) = -^-r r, 
?i-»oo log a„ ^-^ n^ooploga„ aP+^ 

and the proof of (I2.10p is terminated. 

As for the second part of Lemma [221 i.e. equation (|2.11|) . we could proceed 
along similar lines, via the equality 

p k 

^n+l — Z^Z^'^ ^p^k Jn9n 'T'n ^n+l ' 
A:=0 i=0 

with gn = 'Pl.S-^i^Mn and /i„ = M^S-^Mn. ■ 

The proof of theorem 12.11 is completed as relations (|2.10|) and (12. lip are 
clearly direct consequences of (|2.2p and (I2.3|l . respectively. Indeed, since 
I3n ~ dan, we have almost surely 

Vn ~ drun, and /„ ~ d7„, 
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hence convergence (I2.10p immediately yields (12.21) . Moreover in the partic- 
ular case p = 1, the second convergence (|2.1ip is exactly (|2.3|) . Now, for 
p > 2, the elementary expansion 

leads to 

an{p) = {MiS~\MnY - [Ml^S-^MnY 

^^/K-a„(l)xP-i-. (2-21) 



Vn 



= a.(i)yr'E 

Riccati's formula yields 

< {l-fn)tl{S-]_{^^n^iS-]_{^)Vn 

< fnVn, 

hence 0^(1) = o(K) and, by (12211), 

«n.(p) ~ pan(l)K^~^ a.s., 

so that finally 

i^^b^^"^(^)=J^^b^E«'=(lK' =m a.s. (2.22) 

fc=l k=l 



In most of statistical applications encountered so far, the convergence as- 
sumption (|2.ip is satisfied. However, this technical hypothesis somehow cir- 
cumvents the vector problem, which in its full generality is not yet solved. 
Indeed, (|2.ip entails that all eigenvalues of the matrix 5„ grow to infinity at 
the same speed a„. Thus, our method of proof as some features in common 
with the scalar case. Hopefully, we should be able to establish the following 
result, stated for the moment as a conjecture, without assuming (|2.1|) . 

Conjecture 2.3. Let (e„) be a martingale difference sequence satisfying the 
homogeneity condition K[e'^_^_l\J^n] = o"^ o,.s. and assumption (Hp) introduced 
in theorem \2.1l for some integer p > 1. Then, we have 

n 

Yl fk{MiS^\MkY = 0{\ogdn) 

k=l 
n 

Y,{MiS^\Mky - {MlS^'MkY = 0{logdn) a.s. 

k=l 
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3 Applications 

3.1 Linear regression models 

Theorem 12. H is the keystone to understand the asymptotic behavior of cumu- 
lative prediction and estimation errors associated to the stochastic regression 
process given by, for all n > 1, 

Xn+l = e'^n + en+l, (3-1) 

where ^ G M*^ is the unknown parameter. The random variables X„,<^>„,e„ 
are the scalar observation, the regression vector and the scalar driven noise, 
respectively. We propose here two applications. The first one concerns stable 
autoregressive processes while the second one deals with branching processes 
with immigration. 

For a reasonable sequence (On) of estimators of 0, we shall investigate the 
asymptotic performance of O^^n, as a predictor of Xn+i- More precisely, we 
shall focus on the prediction error Xn+i — On^n and on the estimation error 
9n — 0. In fact, it is more relevant and efficient [9] to consider the cumulative 
prediction and estimation errors defined, for some ]5 > 1, as 

n-l 

Cnip) = Y.{Xk+i - 9i<Pk?P (3.2) 

k=0 

and 

n 

Gnip) = Y.kP-'\\0k-9fP. (3.3) 

k=l 

In the scalar case d = 1, under suitable moment conditions, asymptotic 
estimates on (C„(p)) and {Gn{p)) were established in [1] by means of the 
standard least squares (LS) estimator 

n 

^„ = 5^ii^<l>fc-iXfc. (3.4) 

k=l 

It turns out that theorem 12.11 allows us to improve the results of [1,2]. Up 
to our knowledge, only partial results in the particular case p = 1 have been 
obtained, namely in [7,17], where the authors derived the asymptotics of 
(C„(p))and {Gn{p))- For the proofs of the strong consistency of the LS 
estimator for general linear autoregressive model, we refer the reader to 
[8,11,16]. Also, one can find in [8,11,12,16] various results on the asymptotic 
behavior of the empirical estimator of the covariance associated with process 
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One might wonder how the convergence of the moments in the ASCLT helps 
us to deduce the almost sure asymptotic properties of the sequences {Cn{p)) 
and {Gn{p)). It follows from JH]) and that 

9n-e = S-\Mn (3.5) 

where 

n 

Mn = MQ + Y,^k-iek 

k=l 

with Mo = -Se. If 

T^n={e- Onf^n = " ^n^n " En+l, (3.6) 

relations (|3.5p and (|3.6p yield 

rcl = MiS-\^n^iS--iMn. 
Applying now Riccati's formula given e.g. in [7] pages 96 and 99, it comes 

S-\ = + (1 - fn)S-\^n^iS^\. 

Hence, one can write 

a„(l) = M*5-iiM„ - MiS-^Mn = (1 - fn)7^l. (3.7) 

It is often difficult to obtain asymptotic information about the explosion 
coefficient /„. Nevertheless, in our framework, it is possible to show that 
(/n) converges almost surely to zero. So, the asymptotic behavior of (G„(p)) 
and {Cn{p)) can be derived from the properties of (a„(l))^, under some 
suitable moment conditions on the driven noise (en)- By the same token, 
the moments of order 2p can be estimated and controlled. 

Corollary 3.1. Under the assumptions of theorem \2.1{ one has almost surely 
Iim-1-V(a.(l))^ = |^ % (3.8) 

Proof. When p = 1, the convergence (13. 8p corresponds precisely to (12. 3h . 
Suppose now that p > 1. Since 

an(l) < fnVn 

and fn ^ almost surely, we get at once, applying Lemma 12.21 and Kro- 
necker's lemma, 

^ n ^ n 

< hm — - V(afc(l))^ < hm — - V(AFfc)^ = a.s. 

n->oo log dr, n-+oo log dr, 

" k=\ " k=\ 
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3.2 Moment estimation, prediction and estimation errors 

Assume (e„) forms a martingale difference sequence with E[e^_,_]^|^„] = cr^ 
a.s., for all n > 1, and let 

k=l 

If En has a conditional moment of order a > 2, then the strong law of large 
numbers for martingales implies the almost sure convergence of A„ to cr^. 
Under the assumptions of theorem 12.11 with p = 1, the convergence (|3.8|1 
leads to the strong consistency of the estimator of 



^ n— 1 
Tn = - } {Xk+l 



fc=0 
since 

lim — ^(r„- A„) =£7^ a.s. (3.9) 

n.^oo log dn 

Hence, a natural estimator of higher moments of (e„) can be proposed. For 
any integer p> 1, let 



^ n — 1 

Tn{2q) = - Y^i^k+l - ei<^kfP. (3.10) 



A:=0 

One can readily observe that nr„(2p) = Cn{p)- For any integer p > 1, let 

n-l 

n 



^ n— 1 

A„(2p) = -5]4^ (3.11) 



fc=0 

Almost sure asymptotic properties of r„(2p) are given in the next corollary. 

Corollary 3.2. Assume that (e„) satisfies (Hp) with p > 1. In addition, 
suppose that for some integer I < q < p, E[e^'^-^|.7>i] = a{2q) a.s. Then, 
Tn{2q) is a strongly consistent estimator of a{2q) with 

logn^ 



(r„(2g) - A„(2g)) =0(^) (3.12) 



Proof. We already saw via (|3.9p that Corollary 13.21 holds for q = 1. Assume 
now q >2. By expanding the expression of r„(2g), equality (|3.6p leads to 

n— 1 2q—l n— 1 

n(r„(2g) - A„(2g)) = E^'' + E ^.'E^r'^i+i- 

k=0 1=1 k=0 
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We deduce from (|3.7I) together with the almost sure convergence of fn to 
zero and Corollary 13.11 that 

n 
A:=0 

For all £ G {1, . . . , 2g — 1}, let us write 

n-l 



k=0 



with 



n-l 



n-l 



Pn {i) = 4'''^k {i) and Qn {i) = E Trl'^-'ck+i (i) , 



k=0 



k=0 



where an{i) = E[e^_,_;^|^„] and en+i{^) = — o'n(^)- First, since the 
moments <Tn(^) are almost surely bounded, it comes 



n-l 



\Pn{i)\=0{Y,4''~')=(^i'^0gdr, 



a.s. 



k=0 

Moreover, from the estimate 

\Qn{i)\'' = 0{nlogd„ 

we get 



a.s. 



'{rn{2q) - A„(2g))' = 0(nlogd„) a.s.. 



which concludes the proof of Corollary 13.21 ■ 

It is now possible to deduce from Corollary 13.21 the asymptotic behavior of 
(C„(g)). Under the assumptions of Corollary 13.21 the convergence (|3.12p 
yields C„(g)/n converges a.s. to cr{2q). Moreover since the conditional mo- 
ment of order a > 2p of (e^) is almost surely finite. Chow Lemma leads 
to 



k=l 



a.s. 



(3.13) 



for all c such that 2pa ^ < c < 1. Hence it follows from (I3.12p and (I3.13P 
that, if logdn = o(n'^). 



-Cn{q)-a{2q) 
n 



o{n'-') 



a.s. 



Before enoncing the result on the cumulative estimation error {Gn{p)), we 
need another corollary of theorem 12.11 



INRIA 



Convergence of Moments in the ASCLT for Vector Martingales 



17 



Corollary 3.3. Under the assumptions of theorem \2.1{ one has 



lim 

n^oo log dn 



1 

' ^" fc=i 



In addition, assume that it exists a positive definite symmetric matrix L such 
that 

lim — Sn = L a.s. (3.15) 

n— >+co n 

Then, we have 

lim - — V fcp-i ({ek - efL{9k - e))" = e{p) a.s. (3.16) 

n^oo loff n \ J 

Remark 3. Since L is positive definite, (I3.16P immediately yields 

= ©(logn) a.s. 

Proof. Prom the definitions of Sn and of it is easy to see that 

(en-Q)'Sn^n-Q)=Vn+gl. 

Hence, it follows from the convergence (|2.2|) and from the convergence of 
the explosion coefficient (/„) to zero, together with Kronecker's Lemma that 
almost surely 

E - -«))' = .Jim — g A = m. 

Then, the convergence (|3.14p is a straight forward consequence of theo- 
rem [2]T] Using the formel object \/^, we get 



1\P 

a.s. 



Thus, 

lim -^j:F-i((4-e)*i.(4-^))'= lim ^5:^ 

n^oo log n V / n^oo log n Pfc 



a.s. 



The classical Abel transform gives the decomposition 

n p p n-1 p 

E = ir^^-- '^(^ - 1)) - + + <i E ^, (3.17) 
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with 



and 



k=l k=l 



n-1 



Moreover, 



p p 



(S„ - d{n - 1)) - -r^So = o(log dn) a.s. 

Pn PO 



So, it only remains to prove that r„ = o(logn) a.s. Indeed, lemma [2?2] yields 

1 m? 1 1 » ^(p) 

hm > — — = - hm > Tfc^T-i, = , , i a.s. 

n-+oo log n Pfc a n^oo log n C(P+^ 

k=l k=l 

Then, splitting r„ into two terms 

rn = X] ^V^(™fc - "^fc+i) + ^^ 7fc+i"^fc+i, (3.18) 

fc=i '^^^ fc=i '^^^ 

and using the proof of theorem 12.11 together with (|2.12ll . we obtain 

E ^ {KW. - PL.X - - a.s. 

The second term is almost surely o(logn): this is a mere consequence of 
Lemma [22] and of the a.s. convergence of S„ — nd/(5n to zero. Finally, we 
have 

lini V fcf-i ((4 - -0)Y = £{p) a.s. 

n— ►oo log n ^-^ V / 



We shall apply now these asymptotic properties to autoregressive processes 
and to branching processes with immigration, which are both particular cases 
of the general stochastic regression process (|3.ip . 



3.3 The linear autoregressive process 

The linear autoregressive process is defined for all n > 1 by 

d 

^n+\ = 'Y^Qk^n-k+\ +en+l- (3.19) 
k=\ 
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Let C denote the companion matrix associated with (X„ 



C 



( Qx 02 ... 

1 ... 

1 

V ... 



... 

... 



1 



/ 



We shall focus our attention on the stable case which means that we assume 
that p{C) < 1 where p{C) is the spectral radius of the matrix C. In addition, 
we also assume that (e„) is a martingale difference sequence which satisfies 
(Hp) with p>l. If E[e^_,_;^|.7>i] = cr^ a.s. and the matrix 



/I 





a 



\ 




\ ... / 

then convergence (|3.15ll holds with the limiting matrix L given by 



L 



k=0 



(3.20) 



Moreover, one can immediately see that the matrix L is positive definite [7,8]. 
Then we are in a position to state our following result. 

Corollary 3.4. Assume that (e„) satisfies (Hp) with p > 1. In addition, 
suppose that for some integer 1 < q < p, K[s'^l^_^i\j-n] = CF{2q) a.s. Then, 
r„(2(7) is a strongly consistent estimator of a(2q) with 



O 



Tn{2q) - Ani2q) 

Moreover, we also have 
1 " 



log n 



n 



a.s. 



lim 

n^oo log n 



£{p) a.s. 



(3.21) 



(3.22) 



k=i 



3.4 A branching process with immigration 
3.4.1 Estimation of the mean 

Branching process with immigration play an increasingly important role in 
statistical physics, computational biology and evolutionary theory. The con- 
cept of immigration is related to situation in which the population can be 



RR n° 6780 



20 



Bernard Bercu , Peggy Cenac , Guy Fayolle 



enriched by exogenous contributions. The branching process with immigra- 
tion {Xn) is given for all n > 1 by the recursive relation 

Xn 

Xn+1 = ^ Yn^k + In+l, (3.23) 
k=l 

where iYn^k) and {In) are two independent sequences of i.i.d. nonnegative, 
integer-valued random variables. The initial ancestor is Xq = 1. The distri- 
bution of {Yn^k) is commonly called the offspring distribution, while that of 
of {In) is known as the immigration distribution. 

Define 

Wn^k] = m, E[/„] = A, 

var[y„^fc] = o-^, var[/„] = 6^. 

We are interested in the estimation of all the parameters m, X, . Rela- 
tion (|3.23p may be rewritten as the autoregressive form 

Xn+l = mXn + A -f En+l (3.24) 

where = Xn+i — mXn — A. Consequently, the branching process with 
immigration is a particular case of the stochastic regression process given by 
(I3.1|l with = {Xn, 1) and 9^ = {m, A). However, one can observe that the 
situation is a little bit more tricky as (e^) is a martingale difference sequence 
with unbounded conditional variance 

E[4+i|.F„] =a^Xn + b^ 

To circumvent this technical difficulty, we introduce the following regression 
process 

where the random variables Zn+i, and ^„ are given by 

1 /2 ]_ /2 1 /2 

Zn+l = C„ Xn+l, = Cn ^n, = Cn £«+!, 

with 

Cn = Xn + 1. 

Herafter, (^„) is a martingale difference sequence with bounded conditional 
variance E[(^^_,_]^|J?>j] < + b"^ a.s. The mean vector ^* = (m. A) will be 
estimated by the conditional least-squares estimator 

n 

en = s-'Y,^k''^kXk, 

k=l 
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where 



k=0 

In the subcritical case m < 1, Wei and Winnicki [18] have shown the almost 
sure convergence 

limis,. = L a.. 

n—>oo n 

where the hmiting matrix L is given by 



/ E 



L 



E 



x+i 

X 

X+1 



E 



E 



X 



X+l 



x+1 



J 



The notation X stands for a random variable sharing the same distribution 
as the stationary distribution of the Markov chain Consequently, the 

matrix L is positive definite and the following result holds. 

Corollary 3.5. Assume that (e„) satisfies (Hp) with p > 1. In addition, 
suppose that for some integer 1 < q < p, E[e^'^-^|.7>i] = <7{2q) a.s. Then, 
Tn{2q) is a strongly consistent estimator of a{2q) with 



2 / Jos' ft ^ 

Tn{2q) - ^n{2q)] =0{^] a.s. 



n 



Moreover, we also have 



lim 

n— >oo log n 



k=i 



£{p) a.s. 



(3.25) 



(3.26) 



3.4.2 Estimation of the variance 

It follows from equation (|3.24l) that 

el_^^ = a^Xn + b^ + Vn+i, 
where (Vn) is a martingale difference sequence satisfying 

E[V;\i|.F„] = 2a^Xl + Xnir^ - 2>a^ + 45^2) + j/^ - 6^ 

with 

= E[(y„,fc - m)4] and i/^ = E[(4 - A)^]. 
Consequenly, we infer that 
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As before, we estimate the vector of variances rf = ((7^,6^) by the following 
conditional least-squares estimator 

n 

Vn = Qn^ XI with = - ^k^k, 

k=l 

where 

n 

In the subcritical case m < 1, it was established by Winnicki [19] that 

lim —Qn = A 

n— >oo n 

where A is the positive definite limiting matrix given by 





I X2 1 


E 


X 


\ 




[(x+ir\ 


[e 


X 


E 






[(x+ir\ 


[{x+iy\ 



Our last result is the following: 

Corollary 3.6. Assume that (£„) satisfies (Hp) for p > 2. Then 

n 

lini V ((% - 77)*A(% - n)Y = i{p) a.s. (3.27) 



References 

[1] Bercu, B. On the convergence of moments in the almost sure central 
limit theorem for martingales with statistical applications. Stochastic 
Processes and their applications 111 (2004), 157-173. 

[2] Bercu, B., and Fort, J. C. A moment approach for the almost sure 
central limit theorem for martingales. Studia Scientiarum Mathemati- 
carum Hungarica 45, 1 (2008), 139-159. 

[3] Brosamler, G. A. An almost everywhere central limit theorem. Math. 
Proc. Cambridge Philos. Soc. 104 , 3 (1988), 561-574. 

[4] Chaabane, F. Invariance principles with logarithmic averaging for 
martingales. Studia Sci. Math. Hungar. 37, 1-2 (2001), 21-52. 

[5] Chaabane, F., and Maaouia, F. Theoremes limites avec poids pour 
les martingales vectorielles. ESAIM Probab. Statist. 4 (2000), 137-189. 



INRIA 



Convergence of Moments in the ASCLT for Vector Martingales 



23 



[6] Chaabane, F., Maaouia, F., and Touati, A. Generalisation du 
theorems de la limite centrale presque-sur pour les martingales vecto- 
rielles. C. R. Acad. Sci. Paris Ser. I Math. 326, 2 (1998), 229-232. 

[7] DuFLO, M. Random Iterative Methods. Springer- Verlag, 1997. 

[8] DuFLO, M., Senoussi, R., and Touati, A. Proprietes asympto- 
tiques presque sures de I'estimateur des moindres carres d'un modele 
autoregressif vectoriel. Ann. Inst. Henri Poincare 27, 1 (1991), 1-25. 

[9] Goodwin, G., and Sin, K. Adaptative Filtering Prediction and Con- 
trol. Prentice-Hall, Englewood Cliffs, N.J., 1984. 

[10] Lacey, M., and Phillip, W. A note on the almost sure central limit 
theorem. Statist. Probab. Letters 9 (1990), 201-205. 

[11] Lai, T., and Wei, C. Least-squares estimates in stochastic regres- 
sion models with applications to identification and control of dynamic 
systems. The Annals of Statistics 10, 1 (1982), 154-166. 

[12] Lai, T., and Wei, C. Asymptotic properties of general autoregres- 
sive models and strong consistency of least-squares estimates of their 
parameters. Journal of Multivariate Analysis 13 (1983), 1-23. 

[13] LiFSHiTS, M. A. Lecture notes on almost sure limit theorems. Publi- 
cations IRMA 54 (2001), 1-25. 

[14] LiFSHiTS, M. A. Almost sure limit theorem for martingales. In Limit 
theorems in probability and statistics. Janos Bolyai Math. Soc, Bu- 
dapest, 2002, pp. 367-390. 

[15] ScHATTE, P. On strong versions of the central limit theorem. Math. 
Nachr. 137 (1988), 249-256. 

[16] Wei, C. Asymptotic properties of least-squares estimates in stochastic 
regression models. The Annals of Statistics 13, 4 (1985), 1498-1508. 

[17] Wei, C. Adaptative prediction by least squares predictors in stochas- 
tic regression models with applications to time series. The Annals of 
Statistics 15, 4 (1987), 1667-1682. 

[18] Wei, C. Z., and Winnicki, J. Estimation of the means in the branch- 
ing process with immigration. Ann. Statist. 18, 4 (1990), 1757-1773. 

[19] Winnicki, J. Estimation of the variances in the branching process with 
immigration. Probab. Theory Rel. Fields 88, 1 (1991), 77-106. 



RR n° 6780 




Centre de recherche INRIA Paris - Rocquencourt 
Domaine de Voluceau - Rocquencourt - BP 105 - 78153 Le Chesnay Cedex (France) 

Centre de recherche INRIA Bordeaux - Sud Quest : Domaine Universitahe - 351, cours de la Liberation - 33405 Talence Cedex 
Centre de recherche INRIA Grenoble - Rhone-Alpes : 655, avenue de I'Europe - 38334 Montbonnot Saint-lsmier 
Centre de recherche INRIA Lille - Nord Europe : Pare Scientifique de la Haute Borne - 40, avenue Halley - 59650 Villeneuve d'Ascq 
Centre de recherche INRIA Nancy - Grand Est : LORIA, Technopole de Nancy-Brabois - Campus scientifique 
615, rue du Jardin Botanique - BP 101 - 54602 Villers-les-Nancy Cedex 
Centre de recherche INRIA Rennes - Bretagne Adantique : IRISA. Campus universitaire de Beaulieu - 35042 Rennes Cedex 
Centre de recherche INRIA Saclay - Ile-de-France : Pare Orsay Universite - ZAC des Vignes : 4, rue Jacques Monod - 9 1 893 Orsay Cedex 
Centre de recherche INRIA Sophia Antipolis - Mediterranee : 2004, route des Lucioles - BP 93 - 06902 Sophia Antipolis Cedex 



Editeur 

INRIA - Domaine de Voluceau - Rocquencourt, BP 105 - 78153 Le Chesnay Cedex (France) 

http: / /www. inria.fr 
ISSN 0249-6399 



